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Abstract 

Objectives The relatively weak quasi-experimental evaluation design of the original 
Boston Operation Ceasefire left some uncertainty about the size of the program’s effect on 
Boston gang violence in the 1990s and did not provide any direct evidence that Boston 
gangs subjected to the Ceasefire intervention actually changed their offending behaviors. 
Given the policy influence of the Boston Ceasefire experience, a closer examination of the 
intervention’s direct effects on street gang violence is needed. 

Methods A more rigorous quasi-experimental evaluation of a reconstituted Boston 
Ceasefire program used propensity score matching techniques to develop matched treat- 
ment gangs and comparison gangs. Growth-curve regression models were then used to 
estimate the impact of Ceasefire on gun violence trends for the treatment gangs relative to 
comparisons gangs. 

Results This quasi-experimental evaluation revealed that total shootings involving 
Boston gangs subjected to the Operation Ceasefire treatment were reduced by a statisti- 
cally-significant 31 % when compared to total shootings involving matched comparison 
Boston gangs. Supplementary analyses found that the timing of gun violence reductions for 
treatment gangs followed the application of the Ceasefire treatment. 

Conclusions This evaluation provides some much needed evidence on street gang 
behavioral change that was lacking in the original Ceasefire evaluation. A growing body of 
scientific evidence suggests that jurisdictions should adopt focused deterrence strategies to 
control street gang violence problems. 
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Introduction 


Boston received national acclaim for its innovative approach to preventing youth violence 
in the 1990s (see, e.g. Butterfield 1996; Witkin 1997). The well-known Operation Ceasefire 
initiative was an interagency violence prevention program that focused enforcement and 
social service resources on a small number of gang-involved offenders at the heart of the 
city’s youth violence problem (Kennedy et al. 1996). The Ceasefire “pulling levers” 
focused deterrence strategy was associated with a near two-thirds drop in youth homicide 
in the late 1990s (Braga et al. 2001; Piehl et al. 2003) and was soon embraced by the U.S. 
Department of Justice as an effective approach to crime prevention. In his address to the 
American Society of Criminology, former National Institute of Justice Director Jeremy 
Travis (1998) announced “[the] pulling levers hypothesis has made enormous theoretical 
and practical contributions to our thinking about deterrence and the role of the criminal 
justice system in producing safety.” Subsequently, the basic elements of the Boston 
Ceasefire framework has been applied in many American cities through federally spon- 
sored violence prevention programs such as the Strategic Alternatives to Community 
Safety Initiative and Project Safe Neighborhoods (Dalton 2002). 

The evaluation of Boston’s Operation Ceasefire, however, has been greeted with both a 
healthy dose of skepticism (Fagan 2002; Rosenfeld et al. 2005) and some support (Cook 
and Ludwig 2006; Morgan and Winship 2007). The relatively weak quasi-experimental 
evaluation design of the original implementation leaves some uncertainty about the size of 
Ceasefire’s effect on gang violence in Boston and does not provide any direct evidence that 
Boston gangs subjected to the Ceasefire intervention actually changed their offending 
behaviors (Ludwig 2005; Wellford et al. 2005). Given the influence of the Operation 
Ceasefire experience on policing and violence prevention policy, a more rigorous exam- 
ination of the intervention’s effects on street gang behavior in Boston is sorely needed. 

In this paper, we take advantage of unique data on gangs and gang-involved gun 
violence in Boston in a quasi-experimental evaluation of the group-level violence pre- 
vention effects of a reconstituted Operation Ceasefire strategy implemented in 2007. As 
compared to previous evaluations of Operation Ceasefire that focused solely on aggregate 
rates of violence, our quasi-experimental evaluation focuses squarely on the gangs that 
were targeted for treatment. Propensity score matching techniques were used to develop 
matched Ceasefire treatment gangs and comparison gangs. Growth-curve regression 
models were then used to estimate the impact of Ceasefire on gun violence trends for the 
treatment gangs relative to comparisons gangs. We find that the Ceasefire intervention was 
associated with statistically significant reductions in gun violence trends for treatment 
gangs relative to gun violence trends for the comparison gangs. A supplementary analysis 
examined the specific timing of the Ceasefire intervention as applied to each matched 
treatment gang and found that sharp reductions in gun violence immediately followed the 
intervention. 


Literature Review 
The Boston Gun Project and Operation Ceasefire 
The Boston Gun Project was a problem-oriented policing enterprise expressly aimed at 


taking on a serious, large-scale crime problem—homicide victimization among young 
people in Boston. Like many large cities in the United States, Boston experienced a large 
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sudden increase in youth homicide between the late 1980s and early 1990s. The Project 
began in early 1995 and implemented what is now known as the “Operation Ceasefire” 
intervention, which started in the late spring of 1996 (Kennedy et al. 1996). Led by the 
Boston Police Department (BPD), a working group of law enforcement personnel, youth 
workers, and Harvard University researchers diagnosed the youth violence problem in 
Boston as one of patterned, largely vendetta-like (“beef”) hostility amongst a small 
population of chronic offenders, and particularly among those involved in loose, informal, 
mostly neighborhood-based gangs. These gangs represented less than | % of the city’s 
youth between the ages of 14 and 24, but were responsible for more than 60 % of youth 
homicide in Boston. 

The focused deterrence strategy behind Operation Ceasefire was designed to prevent 
violence by reaching out directly to gangs, saying explicitly that violence would no longer 
be tolerated, and backing up that message by “pulling every lever” legally available when 
violence occurred (Kennedy 1997, 2011). The chronic involvement of gang members in a 
wide variety of offenses made them—and their groups—vulnerable to a coordinated 
criminal justice response. The authorities could disrupt street drug activity, focus police 
attention on low-level street crimes such as trespassing and public drinking, serve out- 
standing warrants, cultivate confidential informants for medium- and long-term investi- 
gations of gang activities, deliver strict probation and parole enforcement, seize drug 
proceeds and other assets, ensure stiffer plea bargains and sterner prosecutorial attention, 
request stronger bail terms (and enforce them), and bring potentially severe federal 
investigative and prosecutorial attention to gang-related drug and gun activity. Rather than 
simply dealing with individual offending, groups were held accountable for outbreaks of 
serious gun violence. 

Simultaneously, youth workers, probation and parole officers, and later churches and 
other community groups offered gang members services and other kinds of help. These 
partners also delivered an explicit message that violence was unacceptable to the com- 
munity and that “street” justifications for violence were mistaken. The Ceasefire Working 
Group delivered this message in formal meetings with gang members (known as “forums” 
or “call-ins”), through individual police and probation contacts with gang members, 
through meetings with inmates at secure juvenile facilities in the city, and through gang 
outreach workers. The deterrence message was not a deal with gang members to stop 
violence. Rather, it was a promise to gang members that violent behavior would evoke an 
immediate and intense response. If gangs committed other crimes but refrained from 
violence, the normal workings of police, prosecutors, and the rest of the criminal justice 
system dealt with these matters. But if gang members persisted in their violent behaviors, 
the Working Group concentrated its enforcement actions on their gangs. 

The idea of the Ceasefire “crackdowns” specifically but the focused deterrence model 
more generally was not to eliminate gangs or stop every aspect of gang activity, but rather 
to control and deter serious violence among specified groups (Kennedy 1997). To do this, 
the Working Group explained its actions against targeted gangs to other gangs, as in “this 
gang did violence, we responded with the following actions, and here is how to prevent 
anything similar from happening to you.” The ongoing Working Group process regularly 
watched the city for outbreaks of gang violence and framed any necessary responses in 
accord with the Ceasefire strategy. As the strategy unfolded, the Working Group continued 
communication with gangs and gang members to convey its determination to stop violence, 
to explain its actions to the target population, and to maximize both voluntary compliance 
and the strategy’s deterrent power. 
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Operation Ceasefire Deterrence Mechanisms 


Deterrence theory posits that crimes can be prevented when the costs of committing the 
crime are perceived by the offender to outweigh the benefits (Gibbs 1975; Zimring and 
Hawkins 1973). Most discussions of the deterrence mechanism distinguish between 
“general” and “special” deterrence (Cook 1980). General deterrence is the idea that the 
general population is dissuaded from committing crime when it sees that punishment 
necessarily follows the commission of a crime. Special deterrence involves punishment 
administered to criminals with the intent to discourage them from committing crimes in the 
future. Much of the literature evaluating deterrence focuses on the effect of changing 
certainty, swiftness, and severity of punishment associated with certain acts on the prev- 
alence of those crimes (see, e.g. Apel and Nagin 2011; Blumstein et al. 1978; Cook 1980; 
Nagin 1998; Paternoster 1987). 

In addition to any increases in certainty, severity, and swiftness of sanctions associated 
with gun violence, the Operation Ceasefire strategy sought to gain deterrence through the 
advertising of the law enforcement strategy, and the personalized nature of its application. 
The effective operation of general deterrence is dependent on the communication of 
punishment threats to the public. As Zimring and Hawkins (1973) observe, “the deterrence 
threat may best be viewed as a form of advertising” (p. 142). A key element of the strategy 
was the delivery of a direct and explicit “retail deterrence” message to a relatively small 
target audience regarding what kind of behavior would provoke a special response and 
what that response would be.! 

The available research suggests that deterrent effects are ultimately determined by 
offender perceptions of sanction risk and certainty (Nagin 1998). As described above, 
Operation Ceasefire was targeted on very specific behaviors by a relatively small number 
of chronic offenders who were highly vulnerable to criminal justice sanctions. The 
approach directly confronted violent gang members and informed them that continued 
offending will not be tolerated and how the system will respond to violations of these new 
behavior standards. Face-to-face meetings with offenders are an important first step in 
altering their perceptions about sanction risk (Horney and Marshall 1992; Nagin 1998). As 
McGarrell et al. (2006) suggest, direct communications and affirmative follow-up 
responses are the types of new information that may cause offenders to reassess the risks of 
committing crimes. 

In their recent essay on the limits of lengthy prison stays to deter crime, Durlauf and 
Nagin (2011, p. 40) suggest that “strategies that result in large and visible shifts in 
apprehension risk are most likely to have deterrent effects that are large enough not only to 
reduce crime but also apprehensions.” Focused deterrence strategies, such as Boston’s 
Operation Ceasefire, are identified by Durlauf and Nagin (2011) as having this charac- 
teristic. Moreover, they suggest that these “carrot and stick approaches” to crime pre- 
vention creatively use positive incentives, such as social services and job opportunities, to 
reward compliance and facilitate nonviolent behavior. Durlauf and Nagin (2011) conclude 
their discussion of the promise of focused deterrence strategies with a call for additional 
research and evaluation on the crime reduction benefits of these new approaches. 


' Wright et al. (2004, p. 184) offer a clever metaphor of this perspective: “A restaurant owner can sell more 
prime rib by lowering its price, but not to vegetarian patrons. The price of prime rib here represents the 
situational inducement toward ordering meat, but vegetarianism represents a predisposition away from it, 
and thus the effect of meat pricing significantly varies by level of meat eating.” 
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Evaluation Evidence 


A large reduction in the yearly number of Boston youth homicides followed immediately 
after Operation Ceasefire was implemented in mid-1996. A U.S. Department of Justice 
(DOJ)-sponsored evaluation of Operation Ceasefire revealed that the intervention was 
associated with a 63 % decrease in the monthly number of Boston youth homicides, a 
32 % decrease in the monthly number of shots-fired calls, a 25 % decrease in the monthly 
number of gun assaults, and, in one high-risk police district given special attention in the 
evaluation, a 44 % decrease in the monthly number of youth gun assault incidents (Braga 
et al. 2001). The evaluation also suggested that Boston’s significant youth homicide 
reduction associated with Operation Ceasefire was distinct when compared to youth 
homicide trends in most major U.S. and New England cities (Braga et al. 2001). In a 
companion paper to the main impact evaluation, Piehl et al. (2003) developed an econo- 
metric model that evaluated all possible monthly break points in the time series to identify 
the maximal monthly break point associated with a significant structural change in the 
trajectory of the time series. Controlling for trends and seasonal variations, the timing of 
the “optimal break” in the monthly counts of youth homicides time series was in the 
summer months after Ceasefire was implemented in 1996. 

Given the high profile of the Boston experience, the Ceasefire evaluation has been 
reviewed by a number of researchers and the relationship between the implementation of 
Ceasefire and the trajectory of youth homicide in Boston during the 1990s has been closely 
scrutinized. Fagan (2002) suggested that some of the decrease in homicide may have 
occurred without the Ceasefire intervention in place as violence was decreasing in most 
major U.S. cities. In support of this perspective, Fagan (2002) presented a simple time- 
series graph on youth gun homicide in Boston and in other Massachusetts cities that 
suggested a general downward trend in gun violence may have existed before Ceasefire 
was implemented. Using growth-curve analysis to examine predicted homicide trend data 
for the 95 largest U.S. cities during the 1990s, Rosenfeld et al. (2005) found some evidence 
of a sharper youth homicide drop in Boston than elsewhere but suggest that the small 
number of youth homicide incidents precludes strong conclusions about program effec- 
tiveness based on their statistical models. However, in his review of their analysis, Berk 
(2005) raised a number of statistical and methodological concerns with the analysis 
developed by Rosenfeld and his colleagues. 

Other reviewers, however, have been more supportive of a program effect in their 
reviews of the Ceasefire impact evaluation (see Cook and Ludwig 2006). Ludwig (2005) 
suggested that Ceasefire was associated with a large drop in youth homicide but, given the 
complexities of analyzing city-level homicide trend data, there remained some uncertainty 
about the extent of Ceasefire’s effect on youth violence in Boston. Morgan and Winship’s 
(2007) review of the Ceasefire evaluation concluded that the analysis was a “very high- 
quality example” of how to conduct an interrupted time series analysis of program impact 
and further noted “they offer four types of supplemental analysis ... which can be used to 
strengthen the warrant for causal assertion” (p. 252). 

The National Academies’ Panel on Improving Information and Data on Firearms 
(Wellford et al. 2005) concluded that the Ceasefire evaluation was compelling in associ- 
ating the intervention with the subsequent decline in youth homicide. However, the Panel 
also suggested that many complex factors affect youth homicide trends and it was difficult 
to specify the exact relationship between the Ceasefire intervention and subsequent 
changes in youth offending behaviors. The Panel further observed that the Ceasefire 
evaluation examined aggregate citywide data and did not provide any empirical evidence 
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that treated gangs modified their violent behaviors after being exposed to the intervention. 
In a recent article in The New Yorker (Seabrook 2009, p. 37), well-respected deterrence 
scholar Professor Franklin Zimring echoed the concerns raised by the Panel by stating: 


Ceasefire is more of a theory of treatment rather than a proven strategy ... It’s odd 
that no one has ever said, O.K., here are the youths who were not part of the 
Ceasefire program in Boston, let’s compare them to the youths who were. And no 
one has ever followed up any long range studies of the criminal behavior of the group 
that was in the program, either. We just don’t have the evidence, and until we do we 
can’t evaluate how effective Ceasefire really is. 


The Current Study 


While the existing evidence is strong enough to suggest an association between the 
implementation of Ceasefire and the subsequent drop in Boston youth homicides, we agree 
with the concerns raised by the National Academies’ Panel and Professor Zimring that it is 
difficult to determine whether Ceasefire actually changed violent gang behaviors in Boston 
based on the analysis of aggregate citywide trend data during the 1990s—a period known 
for sudden and surprising decreases in violent crime in the United States (see, e.g. Cook 
and Laub 2002). A more rigorous test of Ceasefire would compare pre-test and post-test 
trends in gun violence outcomes by treated Boston gangs to pre-test and post-test trends in 
gun violence outcomes for an equivalent group of untreated Boston gangs. In this study, we 
take advantage of new data on gangs and gang-involved gun violence in Boston to conduct 
a stronger quasi-experimental evaluation of a reinvigorated version of Operation Ceasefire 
implemented during the late 2000s. 

Despite the national acclaim, the BPD discontinued the Ceasefire strategy as its primary 
response to outbreaks of gang violence in January 2000 (see Braga and Winship 2006). 
Yearly counts of gang homicides, unfortunately, increased linearly after Ceasefire was 
halted in Boston (Braga et al. 2008a). In 1999, the last full year of Ceasefire intervention, 
there were only 5 gang-motivated homicides in Boston. By 2006, this number had 
increased more than seven-fold to 37 gang-motivated homicides in Boston. During this 
time period, the BPD experimented with alternative approaches to violence prevention by 
adapting certain Ceasefire tactics to a broader range of problems such as investigating 
unsolved shootings, facilitating the re-entry of incarcerated violent offenders back into 
high-risk Boston neighborhoods, and addressing criminogenic families in hot spot areas 
(Braga and Winship 2006). Unfortunately, the slate of new approaches seemed to diffuse 
the ability of the City of Boston to deal with gang violence as no one group was focused 
exclusively on addressing ongoing conflicts among street gangs (Braga et al. 2008a). 

At the beginning of December 2006, Edward F. Davis III, former Chief of the Lowell, 
Massachusetts, Police Department, was sworn in by Mayor Thomas M. Menino as the new 
Commissioner of the BPD and was immediately charged with reducing gun violence in the 
city. Drawing on his past experience with a pulling levers strategy to control gang violence 
in Lowell (Braga et al. 2008b), Davis announced that Operation Ceasefire would once 
again be the BPD’s main response to outbreaks of serious gang violence. He promoted 
Gary French, who led many of the BPD’s Ceasefire efforts during the 1990s, to Deputy 
Superintendent with oversight of the Youth Violence Strike Force (YVSF, known infor- 
mally as the “gang unit”), school police unit, and the tactical bicycle unit. With the support 
of Davis and his command staff, French reinstated the Ceasefire approach as a citywide, 
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interagency effort to disrupt ongoing cycles of gang violence. Between January 2007 and 
December 2010, 19 Boston gangs were subjected to the Ceasefire pulling levers focused 
deterrence strategy. 


Analytical Approach 


We used a non-randomized quasi-experimental design to compare serious gun violence 
trends for Boston gangs subjected to the Ceasefire intervention to serious gun violence 
trends for a matched comparison group of Boston gangs that did not receive the Ceasefire 
intervention (Shadish et al. 2002; Rossi et al. 2006). This section describes the develop- 
ment of the data and units of analysis in our quasi-experiment, the identification of 
comparison gangs, and the specification of appropriate statistical models to estimate the 
effect of the Ceasefire intervention on serious gun violence trends for treated gangs relative 
to serious gun violence trends for comparison gangs. 


Data and Units of Analysis 


In this study, we measured serious gun violence by using computerized records of BPD 
official reports of Homicide by Firearm and Assault and Battery by Means of a Deadly 
Weapon—Firearm (ABDW—Firearm) incidents between January 1, 2006 and December 
31, 2010. Incident reports are generated in the BPD by detectives or police officers after an 
initial response to a request for police service. In the State of Massachusetts, ABDW— 
Firearm incidents essentially represent shooting events where guns were fired and victims 
were physically wounded by the fired bullets.” The availability of non-fatal incident data 
has the significant advantage of allowing us to include a wider range of gang-involved gun 
violence. More importantly, the difference between a gun homicide and a non-fatal 
shooting event, as one police officer related to us, “is often only a matter of inches and 
luck—a lot of times a non-fatal shooting is just a failed homicide.” The officer’s sentiment 
suggests that whether or not an event becomes lethal is contingent on several uncontrol- 
lable factors—the aim of the shooter, the distance to the target, a rapid call to the police, 
the response time of medical assistance, and so on. In fact, Zimring’s (1968, 1972) studies 
of wounds inflicted in gun and knife assaults demonstrate considerable overlap between 
fatal and non-fatal attacks and suggest that the difference between life and death is just a 
matter of chance. In the text that follows, we use “shooting” as a term of convenience to 
represent both fatal and non-fatal shooting incidents. 

It is well known that police incident data, such as the Federal Bureau of Investigation’s 
Uniform Crime Reports, have shortcomings. For instance, crime incident data are biased 
by the absence of crimes not reported by citizens to the police and by police decisions not 
to record all crimes reported by citizens (see Black 1970). Although incident reports have 
flaws, careful analyses of these data can yield useful insights on crime (Schneider and 
Wiersema 1990). Moreover, official police incident data are widely used for assessing 
trends and patterns of gun crime (Blumstein 1995; Cook and Laub 2002) and the evalu- 
ation of gun violence reduction programs (see, e.g. Sherman and Rogan 1995; McGarrell 
et al. 2001; Cohen and Ludwig 2003). 


? See Massachusetts General Laws, Chapter 265, Section 15A. 
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To determine whether a shooting event involved a gang member as a suspect, victim, or 
both, the “crime incident review” process was used (see Klofas and Hipple 2006). 
Between 2006 and 2010, the BPD’s Boston Regional Intelligence Center (BRIC) convened 
separate quarterly shooting review meetings for the four policing districts (B-2, B-3, C-11, 
and D-4) that experience the bulk of gun violence in Boston and one quarterly shooting 
review meeting for the remaining policing districts. For each district meeting, detectives 
and officers with detailed knowledge on gangs and gang violence problems were required 
to attend; this included district detectives, plainclothes Anti-Crime district officers, Drug 
Control Unit detectives and officers, Homicide Unit detectives, Special Investigations Unit 
detectives, and YVSF detectives and officers. In each quarterly shooting review meeting, 
BRIC detectives and civilian analysts presented the objective characteristics of each 
shooting event (date, location, victim information, and, if arrested, offender information) 
and the available gang intelligence on the event based on their computerized data systems. 
The meeting participants shared their working knowledge on circumstances of the shooting 
event, the relationships between victims and suspects, and, if the event involved gang 
members, details on the gangs involved in the shooting. 

Researchers attended the quarterly shooting review meetings and partnered with the 
BRIC in collecting, coding, entering, and analyzing the qualitative insights on the nature of 
each shooting event. Figure | presents the yearly counts of gang-involved shootings in 
Boston between 2006 and 2010. Gang-involved shootings were relatively stable between 
2006 (N = 263) and 2007 (N = 253), decreased over the course of 2008 (N = 232) and 
2009 (N = 172), and, despite a small increase over the previous year, remained relatively 
low in 2010 (N = 183). Between 2006 and 2010, gang-involved shootings in Boston 
decreased by 30.4 %. 

The units of analysis in this evaluation are quarterly counts of shootings by and against 
specific Boston gangs between 2006 and 2010. Since shootings by and against any par- 
ticular gang were relatively rare events, we aggregated specific shootings into quarterly 
counts to provide more stable estimates of any measurable impacts of Ceasefire on gang 
shooting behaviors. There were N = 123 gangs in Boston involved in at least one shooting 
between 2006 and 2010. We analyzed three quarterly outcomes for each gang included in 
the evaluation: victim gang-involved shootings, suspect gang-involved shootings, and total 
gang-involved shootings (victim and suspect summed). 
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Fig. 1 Gang-involved shootings in Boston, 2006-2010 
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Matching Treatment Gangs with Comparable Control Gangs 


It is important to note here that evaluating Ceasefire is a particularly difficult task. The 
Ceasefire intervention was explicitly designed to deter continued gun violence by gangs 
not subjected to the treatment. As Kennedy et al. (1996, p. 181) describe in their discussion 
of evaluating Ceasefire in Boston during the 1990s: 


.. rather than trying to protect certain areas or groups from the intervention, as in the 
traditional experimental design, the working group went to considerable effort to 
design an intervention that would create “spillover” effects onto other gangs and 
neighborhoods — through the communications strategy, interfering in active or nas- 
cent gang vendettas, fear reduction, and the like. Thus, a traditional evaluation would 
find no impact—youth homicide would fall in the targeted areas ... and in all other 
areas of the city... 


Kennedy et al. (1997, p. 240) describe how social network analysis concepts were used to 
assist the diffusion of the deterrence message across Boston’s gang landscape: 


We used structural network analysis in pursuit of support for an effective commu- 
nications strategy. Here, [social network analysis software] was employed to identify 
naturally existing subgroups, or “cliques,” such that talking to one member would 
effectively be talking to all members [of that clique] ... for clique identification, 
conflict and alliance networks were combined and analyzed. 


The post-2007 version of Boston Ceasefire attempted to create these spillover effects 
onto other gangs that were socially connected to targeted gangs through rivalries and 
alliances. As Ceasefire interventions were completed on targeted gangs, the Ceasefire 
Working Group directly communicated to their rivals and allies that “they would be next” 
if these groups decided to retaliate against treated rival gangs or continue shootings in 
support of treated allied gangs. These messages were delivered to members of socially- 
connected gangs via individual meetings with gang members under probation supervision 
and through direct “street conversations” with gang members by BPD officers and gang 
outreach workers. 

One key assumption that underlies all controlled program evaluations is the “stable unit 
treatment value assumption” (SUTVA). This assumption requires that the treatment or 
control condition to which a unit is assigned has no impact on the response of another unit 
(Rubin 1990). Including untreated Boston street gangs that were socially connected to 
Ceasefire gangs as comparison groups in our impact evaluation would violate SUTVA. The 
Ceasefire program was explicitly designed to ensure that knowledge of Ceasefire actions 
taken against their immediate rivals and allies would diffuse into these untreated groups 
and influence their subsequent gun violence behaviors. To minimize SUTVA violations, 
we excluded all untreated Boston street gangs that were known to have a rivalry or alliance 
with Ceasefire gangs from consideration as comparison groups in our quasi-experimental 
evaluation. This process resulted in N = 82 gangs that were not socially connected to the 
N = 19 Ceasefire gangs as possible comparison groups.” 


3 We used data from a recent social network analysis of the rivalries and alliances among Boston gangs to 
identify the untreated gangs that were socially connected to the N = 19 Ceasefire gangs. Rivalries and 
alliances between gangs were determined through focus groups with police officers, probation officers, and 
streetworkers (city-employed gang outreach workers) based on their working knowledge of past and 
ongoing gang violence. Some gangs connected in rivalries and alliances to Ceasefire gangs also directly 
received treatment. For instance, the Lucerne Street Doggz had rivalries with eight other gangs and alliances 
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We recognize that our strategy to address possible SUTVA violations is limited. The 
gun violence behaviors of untreated gangs with second- and third-order social connections 
to treated gangs may have been impacted through the indirect transmission of knowledge 
on the consequences experienced by treated gangs. The Ceasefire intervention could have 
also affected the gun violence behaviors of untreated gangs located in proximate neigh- 
borhoods that were not socially connected to treatment gangs through local non-gang 
social networks. In essence, these social dynamics introduce a potential bias against 
establishing a statistically-significant Ceasefire treatment effect. Our analyses would then 
represent a very conservative test of program impacts. 

Using Stata 12.1 statistical software, we executed PSMATCH2 propensity score 
matching routines (Leuven and Sianesi 2003) to develop matched comparison and treat- 
ment groups from the untreated gangs and the Ceasefire gangs. Propensity score matching 
techniques attempt to create equivalent treatment and comparison groups by summarizing 
relevant pre-treatment characteristics of each subject into a single-index variable (the 
propensity score) and then matching subjects in the untreated comparison pool to subjects 
in the treatment group based on values of the single-index variable (Rosenbaum and Rubin 
1983, 1985). As such, we drew upon detailed information on the characteristics of Boston 
gangs from a recent investigation of the relative importance of prior conflicts and the 
proximity of gang turf on gun violence outcomes. The propensity score matching routine 
included the following nine characteristics: 


1. Number of total shootings committed by each gang in 2006 (pre-Ceasefire). Gun 
violence among Boston gangs has been previously described as perpetuated by 
vendettas and ongoing series of retaliations (Kennedy et al. 1996). Gangs with higher 
levels of gun violence have an increased risk of persisting in their shooting behaviors 
over time. 

2. Gang membership size. Gangs with larger memberships have an increased number of 
members who can commit or be victimized by shootings.* 

3. Adjacency to another gang’s turf. Research suggests that gang violence is more likely 
to erupt at the boundaries where gangs’ turf meet (Papachristos 2009; Tita and 
Greenbaum 2009; Tita and Radil 2011). Boston gangs with turf adjacent to the turf of 
another gang are more likely to be involved in serious gun violence.” 

4. Gang longevity. Gangs that have been in existence since the 1990s will have a more 
stable set of rivalries and a longer history of death and injury at the hands of their 


Footnote 3 continued 

with four other gangs. During the study period, three of their rivals (Castlegate, Morse, and Norfolk) and 
three of their allies (Favre, Kaos, and Orchard Park) also experienced Ceasefire interventions. N = 22 
untreated gangs directly connected to Ceasefire gangs via rivalries or alliances were excluded from con- 
sideration for inclusion in our quasi-experimental design. The exercise resulted N = 82 gangs as possible 
comparison groups (123 total gangs—19 treated gangs—22 untreated gangs that were socially connected to 
treated gangs = 82 possible comparison gangs). 


4 Gang membership size was calculated from the roster of members of each gang in the BPD BRIC’s gang 
intelligence database. 


> We used ArcGIS 10.0 mapping software to map the turf of Boston gangs as polygons that occupied a 
circumscribed amount of space. We created a matrix of turf adjacency where a tie occurs if any side of a 
gang polygon touches at least one side of another gang polygon. 
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rivals; the longevity of these gangs and their ongoing disputes with rivals may increase 
the likelihood of a violent dispute during the study period.° 

5. Number of rivalries with other gangs. Gangs with larger numbers of rivalries with 
other gangs have an increased risk that one or more of these rivalries could turn into an 
active violent dispute that would generate a string of retaliatory shootings. Retaliation 
and retribution are perhaps the most frequently cited mechanisms of gang violence 
(Decker 1996; Hughes and Short 2005; Papachristos 2009). 

6. Number of alliances with other gangs. Similar to alliance systems in international 
relations, some gangs form alliances for the benefit of mutual protection. For instance, 
in a unique study on gang finances and strategy in Chicago, Levitt and Venkatesh 
(2000) describe how one gang parlayed and negotiated such alliances to rally other 
groups to their aid during a gang war. 

7. Gang located in housing project. Research has found that housing project areas are 
associated with increased levels of gang homicide relative to other city areas without 
housing projects (Smith 2012). 

8. The concentration of social disadvantage in each gang turf area. We included an 
index that measured concentrated social disadvantage’ in the 2000 US Census block 
group(s) surrounding gang turfs to make certain that comparison gangs were selected 
from neighborhoods that were similar to the neighborhoods in which the Ceasefire 
gangs were located. Research reveals that the degree of concentrated social 
disadvantage in a neighborhood is strongly correlated with the concentration of 
violent crime (Morenoff et al. 2001; Sampson and Wilson 1995) and gang crime in 
these areas (Papachristos and Kirk 2006; Rosenfeld et al. 1999). 

9. Number of gang members arrested in 2006 (pre-Ceasefire). Finally, local police 
departments traditionally use arrest-based enforcement strategies to suppress gang 
violence (Klein 1993). Arrests of gang members could plausibly impact the likelihood 
that a particular gang engages in gun violence through the removal of likely 
“shooters” from the street. 


We recognize that it would have been ideal to include a greater number of covariates in 
our final propensity score matching model. Indeed, the ability to balance treatment and 
comparison groups on as many covariates as possible is the main strength of propensity 
score methods. Unfortunately, these nine covariates represented the only group-level 
descriptors for Boston gangs available at the time of this analysis. Nevertheless, we believe 


© Longevity was determined by comparing the roster of N = 123 gangs with at least one shooting during 
the 2006-2010 study time period to the roster of active Boston gangs in 1995 identified by Kennedy et al. 
(1997). 


7 The concentrated disadvantage index is a standardized index composed of the percentage of residents who 
are black, the percentage of residents receiving public assistance, the percentage of families living below the 
poverty line, the percentage of female-headed households with children under the age of 18, and the 
percentage of unemployed residents (as measured by the percentage of men over the age 16 who did not 
work in the previous year) (see Morenoff et al. 2001; Sampson et al. 1997). Because of the high correlation 
of these variables, we conducted principal components factor analysis, which revealed that variables load on 
a single factor (which was retained as a standardized index variable). For example, a Boston block group 
featuring a disadvantage index score of 1.5 would be 1.5 SD more disadvantaged than the mean Boston 
block group. As such, the disadvantage index is adjusted specifically for the city of Boston using 2000 
Census variables, even while the components used to construct the index remain constant across much 
neighborhood research and remain robust predictors of crime across a variety of city types and spatial 
aggregations. For those gangs whose turf spanned more than one census block group, we used a spatially- 
weighted mean of the connected block groups to calculate the disadvantaged index for the neighborhood 
surrounding each gang’s turf. 
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our parsimonious propensity score model captures the gang-level covariates most directly 
associated with gun violence behaviors that would influence the selection of particular 
gangs for Ceasefire treatment. As we describe in detail below, our impact analysis was not 
affected by unobserved variables that could simultaneously affect gang assignment to the 
Ceasefire treatment and gun violence outcomes. 

The broader propensity score matching literature identifies a wide variety of matching 
algorithms with different choices that need to be made when each approach is used (see, 
e.g. Apel and Sweeten 2010; Heckman et al. 1997; Imbens 2004; Smith and Todd 2005). 
We selected radius matching with a caliper = 0.01 as our primary propensity score 
matching algorithm. According to Dehejia and Wahba (2002), the basic idea of this variant 
is to use not only the nearest neighbor within each caliper but all of the comparison 
members within the caliper. A benefit of radius matching is that the approach uses only as 
many comparison units as are available within the caliper and therefore allows for usage of 
additional units when good matches are available or fewer units when good matches are 
not available (Caliendo and Kopeinig 2005). As such, the approach minimizes the risk of 
bad matches. 

Table 1 reports the results of the propensity score radius matching with a cali- 
per = 0.01. The table presents the pre- and post-matching ¢ tests and the standardized bias 
statistics which represents the mean difference as a percentage of the average standard 
deviation between the groups (Rosenbaum and Rubin 1985). In the matched sample, all 
p values are higher than 0.05, and all bias statistics are less than 20.0 (a general ‘rule of 
thumb’ for balanced groups; see also Austin et al. 2007).° This confirmed that we achieved 
balanced treatment and comparison groups. PPMATCH2 radius matching (caliper = 0.01) 
routine revealed that the 16 matched Ceasefire treatment gangs and 37 matched compar- 
ison gangs were in the common support region. This ensures that gangs with the same 
X values have a positive probability of being both treated and untreated (Heckman et al. 


1999). 
Growth-Curve Regression Model Specification 
We use a variation of a multi-level negative binomial regression model in order to analyze 


the quarterly change in gang-involved shootings for treatment and comparison gangs over 
a 5-year observation period (2006-2010, N = 20 quarters).? More specifically, we 


8 For balancing properties to be satisfied in the propensity score matching analysis, certain pre-treatment 
characteristics needed to be entered as dummy variables into the Stata 12.0 PSMATCH2 routine. The total 
number of shootings in 2006 and the number of members of each gang were entered as interval-level 
measures. Adjacent gang turf was coded “0” for gangs that did not have turf adjacent to another gang’s turf 
and “1” for gangs that did have turf adjacent to another gang’s turf. Longevity was coded “0” for gangs that 
did not exist in 1995 and “1” for gangs that did exist in 1995. The number of rivalries was coded as “0” for 
gangs that had 2 or fewer rivalries and “1” for gangs that had 3 or more rivalries. The number of alliances 
was coded as “0” for gangs that had no alliances and “1” for gangs that had alliances with at least one other 
gang. Housing project gang was coded as “0” for gangs not located in a housing project and “1” for gangs 
were located in a housing project. The concentration of disadvantage in the surrounding Census block 
group(s) was coded as “0” for gang turf located in block groups below the 75th percentile and “1” for gang 
turf located in block groups at the 75th percentile or greater. The number of gang arrests in 2006 was coded 
as “OQ” for gangs with 14 or fewer arrests in 2006 and “1” for gangs with 15 or more arrests in 2006. 


° The quarterly total gang-involved shootings for the N = 53 treatment and comparison gangs used in these 
analyses were distributed as overdispersed count data. The distribution had a mean = 1.39, standard 
deviation = 1.89, and variance = 3.57. One sample Kolmogorov—Smirnov nonparametric tests rejected the 
null hypotheses that the observed distribution was not different from a normal distribution (p < 0.0001) and 
not different from a Poisson distribution (p < 0.0001). 
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Table 1 Balancing treatment and comparison gangs through propensity score matching 


Characteristics Treated Untreated % Bias % Bias reduction t test p>id 


Total shootings in 2006 


Unmatched 8.368 3.683 53.8 2.73** 0.007 

Matched 9.562 8.374 13.6 74.7 0.33 0.746 
Gang size 

Unmatched 41.375 31.920 31.7 1.17 0.247 

Matched 42.267 39.365 9.7 69.3 0.26 0.794 
Adjacent gang turf 

Unmatched 0.526 0.471 10.9 0.44 0.661 

Matched 0.667 0.644 4.4 59.1 0.13 0.901 
Longevity 

Unmatched 0.625 0.540 16.9 0.61 0.543 

Matched 0.600 0.586 2.8 83.6 0.07 0.941 
More than 2 rivalries 

Unmatched 0.577 0.251 69.6 2.95** 0.004 

Matched 0.751 0.699 11.0 84.2 0.27 0.787 
1 or more gang alliances 

Unmatched 0.684 0.416 55.0 2.20* 0.030 

Matched 0.733 0.804 —14.7 73.3 —0.45 0.656 
Housing project gang 

Unmatched 0.211 0.173 9.4 0.39 0.698 

Matched 0.250 0.269 —4.8 48.6 —0.12 0.905 
High disadvantage in census block group 

Unmatched 0.571 0.254 66.4 2.35* 0.021 

Matched 0.500 0.524 —5.1 92.4 —0.11 0.911 
15 or more gang arrests in 2006 

Unmatched 0.578 0.291 59.5 2.51* 0.013 

Matched 0.667 0.675 —18 96.9 —0.05 0.960 


N = 53 (16 treated gangs, 37 comparison gangs) 

Radius matching propensity score model (caliper = 0.01) 
* p < 0.05 

** pn < 0.01 


developed individual growth curve models to estimate street gang changes in violent index 
crime incidents over the observation period (Gelman 2005; Singer and Willet 2003). Here 
we used a longitudinal negative binomial model where we predict within unit variation at 
level 1 and between unit variation at level 2 using level | intercepts and slopes as out- 
comes. In non-technical terms, we are interested in accurately analyzing the overall 
shooting trend of each of the street gangs during the observation period. Each street gang is 
also allowed to have its own slope and intercept in order to model different starting levels 
of shootings as well as different rates of change. This is consistent with the variation 
observed in shootings by gangs—some groups are highly active and others are less active. 

Formally, the model is specified as follows where y;; is the count for the th observation 
in the ith group. The model begins with y;| ~ Poisson(y;,) where y;,| ~ gamma(Aiz, 6;) with 
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Ait = exp(xitP + offset) and 6; represents the dispersion parameter. This produces the 
following equation: 


T(Ait + vir) 1 ‘ é; \™ 
Pr(Vir = yirlxir, 07) = 1 
Yn = yubn 8) =F ro + \T+a) lisa (1) 


Following Gelman (2005) and others (Long and Freese 2006; Singer and Willet 2003), 
this specification yields a negative binomial model for the ith group with dispersion equal 
to 1 + 6, in other words, a constant dispersion within groups. Thus, we feel that such a 
specification fits the observed distribution of our data. 

For a random-effects over-dispersion model, 6 varies randomly across observational 


units. We therefore assume that (45) ~ Beta(r,s). Accordingly, the joint probability of 


the counts for the ith group is: 


i =f [eu — = VirlxXir, 6 if (6 i)d6 
t=1 


_T(r+s) Trt oe An) (st i yin) Fy TO tir) 
T(r)I(s Wertst+ ye Ag ee Lit) p at Ain)T (ir + 1) 
(2) 
For X; = (xj1,...,Xim) and where f is the probability density for 6;. This yields the 
following log likelihood: 


InL= 5 wi aries s) snr $2] nro Sr] —InI(r) = InI(s) 


i=1 k=1 k=1 


inr(: +s ia Sr] + tar (Air + vir) — In (vir + 1} 
(3) 


Pr(Yin =Vi,-- Vin, = = Vin, |X, 


Following these equations, our final model is as follows: 


Yj; = % + B\;(Ceasefire) + B;(period) + f3;(impact) + B4;(trend) + B.5;(trend2) 
+ Bei (quarter2) + B;,(quarter3) + Bg;(quarter4) + Bo;(iptw) (4) 


where the quarterly counts of total gang-involved shooting incidents over the 5-year study 
time period was our primary outcome measure (Y;;). However, in addition to our simple 
effect size analyses, we also analyzed changes in the quarterly counts of victim gang- 
involved shootings and the quarterly counts of suspect gang-involved shootings. To esti- 
mate the effect of the Ceasefire treatment, we created a dichotomous dummy variables 
indicating whether a street gang was in the treatment group (1) or in the comparison group 
(0) (Ceasefire) and whether the quarter was pre-intervention (0) or during the intervention 
period (1) (period). We then created a differences-in-differences (DID) estimator by inter- 
acting these two dummy variables (impact). 

To account for secular linear and nonlinear quarterly trends in the dependent variable, 
we included a variable that was measured as the simple linear additive progression for each 
quarter over the course of the 5-year observation period (trend) and a variable that squared 
this simple linear additive progression for each quarter (trend2). We also controlled for 
seasonal variations in the quarterly counts of shootings by including a polychotomous 
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dummy variable (quarter2, quarter3, and quarter4).'° We estimated the growth curve 
regression models with the inverse-weighted propensity score value (1/p) for each of the 
treatment and comparison gangs (represented in the above equation by iptw). The inclusion 
of this covariate controlled for observable differences between the gangs in the treatment 
and comparison groups given the covariates used to calculate the propensity score (Imbens 
and Wooldredge 2009). 

The XTNBREG command in Stata 12.1 statistical software was used to calculate the 
maximum likelihood estimate of the parameters for the DID estimator and to compute the 
associated probability values; this provided estimates of the effects of the Ceasefire 
intervention on the treatment gangs as relative to the comparison gangs. The parameter 
estimates were expressed as incidence rate ratios (i.e., exponentiated coefficients). Inci- 
dence rate ratios are interpreted as the rate at which things occur; for example, an incidence 
rate ratio of 0.90 would suggest that, controlling for other independent variables, a one unit 
increase in the selected independent variable was associated with a 10 % decrease in the 
rate at which the dependent variable occurs. Following social science convention, the two- 
tailed 0.05 level of significance was selected as the benchmark to reject the null hypothesis 
of “no difference.” 


Results 


Simple Pre-Post Analysis of Matched Ceasefire Gangs and Matched Comparison 
Gangs 


Figure 2 presents the yearly mean total gang-involved shootings between 2006 and 2010 
for the 16 matched Ceasefire gangs and the 37 matched comparison gangs. During the 
study time period, the yearly mean total gang-involved shootings per Ceasefire gang 
decreased by 57.3 % from 9.6 shootings in 2006 to 4.1 shootings in 2010. In contrast, the 
yearly mean total gang-involved shootings per comparison gang decreased by only 20.2 % 
from 8.4 shootings in 2006 to 6.7 shootings in 2010. Consistent with the trends in yearly 
mean total gang-involved shootings, the Ceasefire gangs experienced larger decreases in 
both yearly mean suspect and victim gang-involved shootings relative to the comparison 
gangs. Between 2006 and 2010, yearly mean suspect gang-involved shootings per 
Ceasefire gang decreased by 60.7 % (from 5.6 to 2.2) and yearly mean victim gang- 
involved shootings per Ceasefire gang decreased by 52.5 % (from 4.0 to 1.9); in contrast, 
yearly mean suspect gang-involved shootings per comparison gang decreased by 23.3 % 
(from 4.3 to 3.3) and yearly mean victim gang-involved shootings per comparison gang 
decreased by 17.1 % (from 4.1 to 3.4). 

Standardized mean difference effect size statistics were used to determine whether the 
shooting reductions observed for the treated Ceasefire gangs were significantly larger than 
the shooting reductions observed for the comparison gangs. The standardized mean-dif- 
ference effect size (d) is designed for contrasting two groups on a continuous dependent 
variable (Lipsey and Wilson 2001). For this simple analysis, we calculated the mean Time 


'© Quarter 1 served as the reference category for this polychotomous dummy variable. Quarter 1 represented 
whether the outcome included the sum of January, February, and March shootings (1 = Yes, 0 = No). 
Quarter 2 represented whether the outcome included the sum of April, May, and June shootings (1 = Yes, 
0 = No). Quarter 3 represented whether the outcome included the sum of July, August, and September 
shootings (1 = Yes, 0 = No). Quarter 4 represented whether the outcome included the sum of October, 
November, and December shootings (1 = Yes, 0 = No). 
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Mean Total Shootings Per Gang 
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—— Matched Ceasefire Gangs (N=16) —# Matched Comparison Gangs (N=37) 


Fig. 2. Mean gang-involved shootings for matched ceasefire gangs and matched comparison gangs 


2 (year 2010) minus Time 1| (year 2006) gain score, the SD of the gain score, and the 
correlation between the Time | and Time 2 scores for the matched 16 Ceasefire gangs and 
the 37 matched comparison gangs. These statistics were entered into David B. Wilson’s 
Practical Meta-Analysis Effect Size Calculator to estimate the standard mean difference 
effect sizes.'' For total gang-involved shootings, the Ceasefire intervention was associated 
with a large, statistically-significant standardized mean difference effect size favoring 
treatment conditions over control conditions (d = —0.7678; 95 % CI = —1.4221, 
—0.1136; v = 0.1114). For suspect gang-involved shootings, the Ceasefire intervention 
was associated with a larger statistically-significant standardized mean difference effect 
size favoring treatment conditions over control conditions (d = —0.869; 95 % CI = 
—1.6022, —0.1358; v = 0.1339). While the statistic suggested a beneficial impact on 
victim gang-involved shootings, the standardized mean difference effect size was modest 
and not statistically significant (d = —0.4799; 95 % CI = —1.1807, 0.2209; v = 0.1278). 


Growth Curve Regression Model and Sensitivity Analysis Results 


Table 2 presents the results of the growth curve regression models. Controlling for the 
other covariates, the Ceasefire intervention was associated with a statistically-significant 
30.8 % reduction (p < 0.05) in quarterly total gang-involved shootings, a statistically- 
significant 34.7 % reduction (p < 0.05) in quarterly suspect gang-involved shootings, and 
a Statistically-significant 26.9 % reduction (p < 0.05) in quarterly victim gang-involved 
shootings for the treatment gangs relative to the comparison gangs. The Ceasefire dummy 
variable was not statistically significant (p < 0.05) for all three outcome variables, con- 
firming that the matched groups were comparable on the gun violence outcome measures 
controlling for the other covariates. For all three outcome variables, the growth curve 
regression models revealed that Boston gang-involved shootings had statistically-sig- 
nificant seasonal variations; relative to January through March quarterly gang-involved 
shooting counts (Quarter 1), April through June (Quarter 2) and July through September 
(Quarter 3) experienced higher counts of gang-involved shootings (p < 0.01). As expected, 
the inverse propensity score had a statistically-significant negative association with the 


" http://gemini.gmu.edu/cebep/EffectSizeCalculator/d/mean-gains-scores-and-gain-score.html. 
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Table 2 Ceasefire impacts on gang-involved shooting incidents: growth curve regression models 


Ceasefire impact (interaction) 
Ceasefire gang (1 = treated) 


Period (1 = intervention) 


Shooting suspect 


0.653 (0.117)* 
1.167 (0.139) 
0.681 (0.173) 


Shooting victim 


0.731 (0.101)* 
1.031 (0.098) 
0.816 (0.211) 


Total shooting 


0.692 (0.108)* 
1.099 (0.109) 
0.756 (0.152) 


Trend 0.917 (0.055) 0.993 (0.063) 0.940 (0.045) 
Trend-squared 1.004 (0.002) 0.999 (0.003) 1.002 (0.002) 
Quarter 2 1.504 (0.181)** 1.364 (0.177)* 1.463 (0.143)** 
Quarter 3 1.389 (0.177)** 1.376 (0.187)* 1.401 (0.145)** 
Quarter 4 0.763 (0.115) 0.982 (0.149) 0.859 (0.102) 


Inverse propensity score 


0.984 (0.007)** 


0.976 (0.007)** 


0.978 (0.006)** 


Constant 3.495 (1.001)** 2.644 (0.787)* 3.275 (0.722)** 
Log likelihood —1,123.031 —1,069.628 —1,552.587 
Wald x? 108.12** 58.82** 123.36** 

Wald df 9 9 9 

Observations (gangs x quarters) 1,060 1,060 1,060 

Number of gangs 53 53 53 


Coefficients expressed as incidence rate ratios. SE are in parentheses. Quarter 1 is the reference category for 
the seasonal dummy variable 


* p <0.05 
* p< 0.01 


three gang-involved shooting outcome variables (p < 0.01).'? This suggests that Boston 
gangs with higher levels of shootings were more likely to be included in the quasi- 
experimental analysis. 

While the propensity score matching process ensures balance on observed confounders, 
unobserved variables could simultaneously affect assignment into treatment and the out- 
come (Rosenbaum 2002). This hidden bias would alter our inferences about Ceasefire 
treatment effects. For instance, our propensity score model did not include information 
about the organizational structure of Boston gangs. A recent study by Decker et al. (2008) 
demonstrates that even modest increases in organizational structure are correlated with 
increases in patterns of victimization and offending. To examine the robustness of our 
results against possible hidden bias, we used the bounding approach proposed by 
Rosenbaum (2002) via the RBOUNDS user-written routine in Stata 12.1 (DiPrete and 
Gangl 2004). The Rosenbaum bounds techniques allows researchers to determine how 
strongly an unobserved variable must influence the selection process to alter inference 
about treatment effects. No hidden bias is represented when bound estimate T= 1. 


'2 Since the selection of a matching algorithm and its particular specification can be a subjective process 
(Apel and Sweeten 2010), we conducted a supplementary analysis to ensure that any program impacts were 
robust across a variety of matching algorithms and caliper/bandwidth selections. This exercise was not 
intended to be an exhaustive examination of all possible propensity score methods. As such, we included a 
representative selection of approaches: radius matching (calipers = 0.1, 0.01, 0.001), Gaussian kernel 
matching (bandwidth = 0.1, 0.01, 0.001), Epanechnikov kernel matching (bandwidth = 0.1, 0.01, 0.001), 
stratification matching (10 strata), and simple nearest neighbor matching. While the estimates differed 
somewhat across the varying propensity score matching methods, the Ceasefire treatment effect remained 
robust. The Ceasefire impact estimates ranged from a statistically significant 28 % reduction (p < 0.05) toa 
statistically significant 35 % reduction (p < 0.05). 
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Table 3 Sensitivity analysis: 


Rosenhausi bands for Céasehire Ceasefire treatment effect T p-critical me bias 
treatment effect ii a 
Total gang-involved shootings 1.40 0.050 —4.41 
1.45 0.055 —4.57 
Gang suspect shootings 1.50 0.048 —2.52 
Hidden bias equivalents were 155 0.054 —2.68 
calculated at the mean of the Gang victim shootings 1.35 0.046 =1.73 
observed covariates for 2010 1.40 0.053 _1.88 


gang shootings 


Underestimated or overestimated treatment effects that may be due to unobserved con- 
founding are represented by I’ bound estimates higher than 1. A scenario of T = 1.50 
suggests that hidden bias would increase the odds of receiving Ceasefire treatment for 
gangs actually receiving Ceasefire treatment by 50 % relative to gangs that did not receive 
Ceasefire treatment. 

Table 3 presents the results of our Rosenbaum bounds sensitivity analysis. Given the 
direction of the estimated Ceasefire treatment effect, our analysis focused on negative self- 
selection of gangs into the treatment. Positive self-selection of gangs would simply cause 
our findings to be conservative. The p-critical values represent the bound of the signifi- 
cance level of the treatment effect in the case of endogenous selection into treatment status 
(DiPrete and Gang] 2004). The results show that the critical level of I’ at which the 
estimated Ceasefire treatment effect would no longer be statistically significant at the 5 % 
level is 1.45 for total gang shootings, 1.55 for suspect gang shootings, and 1.40 for victim 
gang shootings.'* Our conclusion that gun violence involving gangs that received the 
Ceasefire treatment was significantly lower than gun violence involving gangs that did not 
receive Ceasefire treatment would be challenged if an unobserved variable increased the 
odds that Ceasefire gangs received the Ceasefire treatment by 45 % for total shootings, by 
55 % for suspect shootings, and by 40 % for victim shootings. 

Table 3 also presents the magnitude of the hidden bias that would cause us to revise our 
findings of the causal effects of Ceasefire on gang shootings. Hidden bias equivalents were 
calculated at the mean of the covariates for 2010 gang shootings. For total gang shootings 
in 2010, the critical level of [ = 1.45 is attained at a difference of 4.57 shootings per gang. 
The Ceasefire average treatment effect on treated (ATT) is —5.21 (SE = 2.41, p < 0.05) 
for 2010 total gang shootings.'* The unobserved variable would have to produce a dif- 
ference of similar magnitude to the Ceasefire treatment effect in order to alter our con- 
clusions. While these results convey important information about the level of uncertainty 
contained in matching estimators by showing how large a confounding variable must be to 
undermine the conclusions of our matching analysis, it is important to note that Rosenbaum 
bounds represent a “worst case” scenario (DiPrete and Gangl 2004). As such, these 


'3 A value of T = 1.45 for total gang shootings indicates that the confidence interval for the Ceasefire 
treatment effect would include zero if an unobserved variable caused the chance of treatment assignment to 
differ between treatment and control groups by 1.45 and if this variable’s effect on total shootings was so 
strong as to almost perfectly determine whether total shootings would be bigger for the treatment or the 
control gang in each pair of matched gangs in the data (see DiPrete and Gang] 2004). 


'4 Similar conclusions can be drawn by comparing the Rosenbaum bounds results to the ATT models for 
2010 suspect shootings (ATT = —3.54, SE = 1.73, p < 0.05) and 2010 victim shootings (ATT = —2.11, 
SE = 1.24, p < 0.10). For all three ATT models (radius matching, caliper = 0.01), bootstrapped standard 
errors with 100 replications are provided. 
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analyses suggest that our propensity score matching estimators are robust to hidden bias 
caused by an unobserved confounder. 


Supplementary Analysis of the Timing of Treatment and 
Observed Reductions in Gang Shootings 


We selected January 2007 as the start date of the reinvigorated Operation Ceasefire 
strategy because it represented the first full month of a regime change in the BPD that 
delivered a fully-implemented program. Given the complex and intensive work required to 
implement a focused deterrence intervention on an individual gang, it was simply not 
possible for the Ceasefire Working Group to address the persistent violent behavior gen- 
erated by all treated gangs at the same point in time. The Ceasefire intervention was 
applied to 9 gangs in 2007, 6 gangs in 2008, and 1 gang in 2009. As such, the actual 
delivery of the intervention to treated gangs occurred in a staggered manner during the 
post-intervention time period. The overall dosage of Ceasefire intervention to Boston gangs 
increased during the post-intervention period as suggested by the linear decrease in yearly 
total shootings by treated gangs in Fig. 2. 

To make a direct link between the application of the Ceasefire treatment and subsequent 
changes in violent gang behavior, we conducted an exploratory analysis to identify abrupt 
statistically-significant reductions, known as structural breakpoints, in quarterly total gang- 
involved shootings for each of the 16 matched Ceasefire gangs. Using the NBREG com- 
mand in Stata 12.1, we ran a series of 18 negative binomial regressions for each Ceasefire 
gang with a varying quarterly intervention point between Quarter 2 and Quarter 19 that 
included controls for secular trends and quarterly seasonal variations. Dummy variables 
(0 = pre-intervention, 1 = intervention) were used to estimate the adjusted pre-post mean 
difference in total shootings by and against each Ceasefire gang for each of the 18 quarters 
between Quarter 2 and Quarter 19.'° A sharp and sustained break in the quarterly shooting 
time series will lead to significant before and after differences for several time periods 
around the intervention. This is because these structural breakpoint analyses involve, in 
essence, comparisons of two means adjusted for other factors (see Piehl et al. 2003). 
However, if Ceasefire did produce the desired impact, the maximal structural breakpoint in 
each time series should coincide with the quarter when treatment was applied or in the 
quarter immediately following the treatment application. 

We reviewed official records maintained by the BPD on Ceasefire actions during the 
study time period to determine the specific quarter that the treatment was fully imple- 
mented. Ceasefire was considered fully implemented for a targeted gang when three 
components were present: (1) direct communications with the gang had occurred, (2) social 
services and opportunities were available to gang members who wanted them, and (3) a 
customized law enforcement response was delivered. We illustrate our structural break- 
point analyses by presenting the details of this exercise for the first gang to receive the full 
Ceasefire treatment under the new regime. 

The Lucerne Street Doggz was the first group selected for Ceasefire intervention 
because it was the most violent gang in Boston at the beginning of the study time period. 
The Doggz were a loosely-organized gang based in the disadvantaged Lucerne Street area 
of the Mattapan section of Boston (District B-3). In 2006, the Lucerne gang had roughly 50 
members and was involved in violent disputes with eight rival gangs—Big Head Boys, 


'S We excluded Quarter 1 and Quarter 20 to ensure that our quarterly impact estimates were based on at 
least two quarters (6 months) of shooting data for each Ceasefire gang. 
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Morse Street, Norfolk, Greenwood, Heath Street, Orchard Park, H-Block, and Winston 
Road. Lucerne was the suspect group in 30 gang-involved shootings and the victim group 
in 7 gang-involved shootings in 2006. BRIC intelligence suggested that most of the 
Lucerne shootings, which accounted for nearly 10 % of all Boston shootings in 2006, were 
carried out by no more than 6 or 7 members of the gang. 

In late 2006, BPD District B-3 detectives and officers decided to implement a Ceasefire 
intervention to address the persistent shootings generated by Lucerne. They partnered with 
the U.S. Attorney’s Office, Suffolk County District Attorney’s Office, Boston School 
Police, Massachusetts Department of Youth Services, Massachusetts Department of Pro- 
bation, Boston Ten Point Coalition, Boston Centers for Youth and Families streetworkers, 
Youth Service Providers Network (social work program) and Youth Opportunities Boston 
(non-profit employment development agency) on a “call-in” to deliver the Ceasefire anti- 
violence message. On November 14, 2006, 22 members of the Lucerne Street Doggz 
attended the call-in; 11 members made appointments with Youth Opportunities Boston to 
explore job placement options and 7 members requested follow-up meetings with Youth 
Service Providers Network counselors. Unfortunately, since the BPD was not fully 
invested in the Ceasefire approach, Lucerne did not face any enhanced enforcement 
response to their continued violent behavior after the call-in. BPD participation in the 
Lucerne Street effort was limited to a handful of B-3 detectives and officers; the citywide 
YVSF and the Drug Control Unit were not involved in this initiative. After a relatively 
quiet winter period, Lucerne continued its torrid involvement in shootings and, by the end 
of May 2007, was the suspect group in another 21 gang-involved shootings and the victim 
group in another 6 gang-involved shootings. 

As described earlier, in December 2006, newly-appointed Commissioner Davis man- 
dated that Ceasefire needed to be the BPD’s marquee response to ongoing gang violence. 
In January 2007, then-Deputy Superintendent Gary French, who was charged by Davis to 
coordinate the citywide implementation of Ceasefire, started regular meetings of the 
interagency Operation Ceasefire working group. It was critical to establish the credibility 
of the Ceasefire anti-violence message on the streets of Boston again. Since Lucerne had 
been subjected to a call-in and continued on its violent path, the Ceasefire working group 
needed to make good on the promise that a strong enforcement response would soon 
follow. With the support of the Drug Control Unit and District B-3 personnel, the YVSF 
worked with the U.S. Attorney’s Office, Suffolk County District Attorney’s Office, Drug 
Enforcement Administration and Bureau of Alcohol, Tobacco, Firearms, and Explosives in 
a focused investigation of the Lucerne Street Doggz. On May 24, 2007, 25 Lucerne Street 
gang members were taken into custody and charged with federal and state drug and 
firearms offenses (Ellement 2007). As Fig. 3 reveals, the impact of the Ceasefire inter- 
vention on their gun violence behavior was noteworthy. In 2006 and 2007, Lucerne gang 
averaged 33.5 total shootings per year. Their yearly average plummeted by 87.2 % to 4.3 
per year between 2008 and 2010. 

Table 4 presents a summary assessment of the timing of Ceasefire interventions and 
maximum quarterly total shooting reductions for the 16 matched treatment gangs. Since 
this was an exploratory analysis of only 20 quarterly observations for each gang, we 
relaxed our benchmark to reject the null hypothesis of “no difference” to the less 
restrictive p < 0.10 level. The key components of Ceasefire intervention on the Lucerne 
Street Doggz—direct communications with the gang, offers of services and opportunities, 
and the delivery of an enhanced enforcement response—were in place in Quarter 6 (April— 
June 2007). The table shows that the maximum statistically-significant reduction 
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Number of Shootings 


Fig. 3 Total shootings involving Lucerne Street Doggz, 2006-2010 
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Table 4 The timing of ceasefire interventions and maximum shooting reductions for 16 matched treatment 


gangs 


Treatment gangs 


Lucerne 

Morse 

Favre 

Norfolk 

Kaos 

Castlegate 
Everton/Geneva 
Greenfield 
Heath 

St. James 
H-Block 

Wood Ave 
Orchard Park 
Forest Hills 
Wainwright Park 


Annunciation/mission 


Ceasefire quarter 


Apr—Jun 07 
Apr—Jun 07 
Apr—Jun 07 
Jul-Sep 07 
Jul-Sep 07 
Jul-Sep 07 
Jul-Sep 07 
Jul-Sep 07 
Jul-Sep 07 
Jan—Mar 08 
Jan—Mar 08 
Jan—Mar 08 
Apr—Jun 08 
Jul-Sep 08 
Oct-Dec 08 
Apr-Jun 09 


Max. reduction quarter 


Jul-Sep 07 
Jul-Sep 07 
Jul-Sep 07 
Jul-Sep 07 
Jan—Mar 08 
Oct—Dec 07 
Oct-Dec 07 
Jul-Sep 07 
Jul-Sep 07 
Apr-Jun 08 
Apr-Jun 08 
Apr-Jun 08 
Jul-Sep 08 
Oct-Dec 08 
Apr-Jun 09 
Jul-Sep 09 


Ceasefire coef. (SE) 


—0.65 (0.30)* 
—0.71 (0.48)~ 
—1.29 (0.58)* 
—0.73 (0.29)** 
—0.53 (0.98) 
—3.00 (1.20)** 
—2.39 (0.82)** 
—1.94 (0.85)** 
—0.83 (0.42)* 
—1.04 (0.47)* 
—0.54 (0.31)T 
—0.44 (0.28) 
—0.75 (0.41) 
—0.51 (0.30)T 
—0.87 (0.59) 
—3.19 (1.03)** 


Effect? 


N = 20 quarters per gang 


Negative binomial regression models controlling for simple linear trends and seasonal variations were used 
to identify the maximal break point in each of the 16 time series. The models suggested a statistically- 
significant Ceasefire impact in 13 of the 16 gang-involved total shooting time series (binomial sign test 
proportion = 0.8125, two-tailed p = 0.0213) 


+ p <0.10 
* p < 0.05 
** pn < 0.01 
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(p < 0.05) in the quarterly counts of total shootings for Lucerne occurred in Quarter 7 
(Jul-Sep 07) of the time series. 

As Table 4 reveals, 13 of the 16 matched treatment gangs experienced their largest 
statistically-significant reduction in total shootings in the same quarter as or the quarter 
immediately following the full implementation of Ceasefire. To test whether this distri- 
bution of “successes” relative to “failures” was significantly different than what would be 
expected by chance, we used an application of the binomial distribution known as the sign 
test (Blalock 1979). This test examines the probabilities of getting an observed proportion 
of successes from a population of equal proportions of successes and failures. The 
observed distribution binomial sign test proportion = 0.8125 (13/16) with a two-tailed 
p = 0.0213. This suggests that the observed relationship between the implementation of 
Ceasefire and the timing of the largest statistically-significant reductions was not generated 
by a random process. In other words, Ceasefire generated noteworthy changes in the gun 
violence behaviors of targeted gangs during the post-intervention time period. 


Conclusions 


There is a growing body of evidence that focused deterrence strategies, such as the pulling 
levers approach pioneered by Operation Ceasefire in Boston, generate significant crime 
reduction benefits. A recently completed Campbell Collaboration review of 11 controlled 
evaluations found that focused deterrence strategies were associated an overall statistically 
significant, medium-sized crime reduction effect (Braga and Weisburd 2012). This review 
considered replications of the Boston Ceasefire program in five other jurisdictions, 
including Cincinnati (Engel et al. 2011), Indianapolis (Corsaro and McGarrell 2009; 
McGarrell et al. 2006), and Los Angeles (Tita et al. 2004). Indeed, the available scientific 
evidence suggests that cities suffering from gang and criminally-active group violence 
should experiment with pulling levers focused deterrence strategies. 

Our quasi-experimental evaluation estimated that the reconstituted Boston Ceasefire 
intervention generated a 31 % reduction in total shootings for treated gangs relative to total 
shootings for matched comparison gangs. Relative to matched comparison gangs, matched 
treatment gangs committed significantly fewer shootings and experienced significantly 
lower levels of violent gun victimization. However, it is important note that this evaluation 
yielded a much more conservative violence reduction estimate when compared to the two- 
thirds reductions in youth homicides reported in the original Ceasefire quasi-experimental 
evaluation (Braga et al. 2001; Piehl et al. 2003). While the biases in quasi-experimental 
research are not clear (e.g. Campbell and Boruch 1975; Wilkinson and Task Force on 
Statistical Inference 1999), recent reviews in crime and justice suggest that weaker 
research designs often lead to more positive outcomes (e.g. see Weisburd et al. 2001; 
Welsh et al. 2011).'° 


‘6 Using the Maryland Scientific Methods Scale (Sherman et al. 1997) as a standard, the original Ceasefire 
impact evaluation would be considered a “Level 3” evaluation and also regarded as the minimum design that is 
adequate for drawing conclusions about program effectiveness. This design rules out many threats to internal 
validity such as history, maturation/trends, instrumentation, testing, and mortality. However, as Farrington 
et al. (2002) observe, the main problems of Level 3 evaluations center on selection effects and regression to the 
mean due to the non-equivalence of treatment and control conditions. This evaluation of Ceasefire would be 
considered a “Level 4” evaluation as it measures outcomes before and after the program in multiple treatment 
and control condition units. These types of designs have better statistical control of extraneous influences on the 
outcome and, relative to lower level evaluations, deals with selection and regression threats more adequately. 
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Importantly, this study also provides some much needed evidence to address some of 
the well-thought out concerns over the original Boston Ceasefire evaluation raised by the 
National Academies’ Panel on Improving Information and Data on Firearms (Wellford 
et al. 2005) and by Professor Zimring. Our analyses showed that Boston gangs subjected to 
the post-2007 Ceasefire treatment did indeed change their gun violence behaviors relative 
to Boston gangs that did not receive Ceasefire treatment. Our study also represents an 
important advance over other focused deterrence evaluations that examined aggregate 
citywide changes in group behavior. In Indianapolis (Corsaro and McGarrell 2009) and 
Cincinnati (Engel et al. 2011), evaluators compared citywide gang and criminally-active 
group homicide trends, respectively, to citywide non-gang and non-criminally-active group 
homicide trends, respectively. These evaluations did not distinguish post-intervention 
homicide trends for treated groups relative to post-intervention homicide trends for 
untreated groups. 

Some readers may wonder whether this evaluation can comment on the “true” impact 
of Ceasefire on serious gun violence in Boston. Indeed, this evaluation focused on 
addressing a key question posed by the National Academies’ Panel and Zimring—whether 
treated Ceasefire gangs actually changed their violent behavior. Kennedy (1997), however, 
suggests that the Ceasefire focused deterrence strategy was intentionally designed to deter 
the violent behavior of gangs not directly exposed to the intervention. In essence, our 
statistical models estimated the effect of treatment on the “directly” treated gangs but not 
on the “indirectly” treated gangs. A full accounting of Ceasefire violence reduction effects 
in Boston would also examine these second-order impacts. An important avenue of future 
research would be to determine whether focused deterrence strategies created “spillover” 
violence reduction effects onto other gangs and neighborhoods. Indeed, building upon this 
study, we are pursuing analyses to examine whether untreated gangs changed their gun 
violence behaviors after their rivals and/or allies were subjected to the Ceasefire 
intervention. 

The available research on Ceasefire and its replications has thus far provided scant 
empirical evidence on the ways individuals nested within targeted groups and social net- 
works may change their criminal decision making processes. The Ceasefire mechanism of 
putting gangs “on notice” is designed to increase the certainty of punishment for the group 
as a whole, but it does so through (a) the diffusion of the message among individual group 
members and (b) reliance on the group members, as a collective, to modify behavior 
accordingly. Unfortunately, our study was not able to analyze data on individual behavior. 
However, the next generation of research on focused deterrence strategies should take 
advantage of an important opportunity to understand how changing the certainty of pun- 
ishment for group-level criminal activity may affect individual as well as group behavior. 
A recent study by Loughran et al. (2011b) offers evidence of a “tipping” effect, whereby 
perceived risk deters only when it reaches a certain threshold, and a substantially accel- 
erated deterrent effect for individuals at the high end of the risk continuum (Loughran et al. 
2011b). Yet, another study found diminishing ambiguity of certainty had no observed 
deterrent effect for crimes involving contact between offender and victim (Loughran et al. 
2011a). It is possible that Ceasefire’s unambiguous face-to-face meetings with gang 
members, coupled with demonstrated increases in the swiftness, certainty, and severity of 
punishment for gun violence, exceeds the threshold for a tipping effect and substantially 
accelerates the deterrent effect of the intervention for high-risk gang members. Or, it is 
possible that some such tipping points operate in unexplored ways when collectivities such 
as gangs are involved. Future research on Ceasefire-like interventions would do well to 
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consider how individual decision making processes operate in the context of group 
accountability and interventions. 

While determining whether a program generates the desired outcomes remains an 
important task, we strongly believe that the next wave of research on focused deterrence 
strategies needs to understand why these strategies seem to work and how these strategies 
can be sustained over time. A growing number of scholars suggest that that there seems to 
be additional crime control mechanisms at work in these strategies beyond straight-up 
deterrence (Braga 2012; Corsaro et al. 2012; Papachristos et al. 2007). Other prevention 
frameworks, such as community social control and procedural fairness, might help explain 
the observed impacts of focused deterrence programs on crime. There is also a growing 
body of literature suggesting that it is very difficult in practice to sustain these initiatives 
over an extended time period. Beyond the cessation of Ceasefire in Boston noted earlier, 
replication programs in Baltimore and Minneapolis unraveled rapidly after some encour- 
aging initial crime control success stories (see Kennedy 2011). The Cincinnati Initiative to 
Reduce Violence, however, has been able to institutionalize and sustain its focused 
deterrence interventions through the establishment of a comprehensive organizational 
structure and a governing board (Engel et al. 2011). Clearly, jurisdictions interested in 
implementing focused deterrence strategies need to understand how to keep these programs 
on track for the long-term. 
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